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EXECUTIVE  SUMMARY 


Background  and  Problem 

Management  and  control  of  Army  field  exercises,  as  well  as  re¬ 
alistic  methods  for  assessing  unit  proficiency  through  such  exercises, 
represents  a  current  concern  to  those  responsible  for  establishing  and 
maintaining  combat  readiness.  Realistic  and  meaningful  field  exercise 
management,  control,  and  assessment  requires  technologically  advanc¬ 
ed  control  and  communication  systems.  The  conceptual  design  for  such 
an  exercise  monitoring  and  reporting  system  (EMARS)  was  previously 
developed  but  was  not  subjected  to  empirical  evaluation. 

A  computer  model,  NETMAN,.  for  evaluating  such  systems  was 
also  previously  developed,  tested  for  sensitivity,  and  validated.  How¬ 
ever,  the  adaptability/flexibility  of  NETMAN  was  never  investigated. 
The  present  work  allowed  such  a  test  by  applying  NETMAN  to  the  con¬ 
ceptual  EMARS'  design  and  evaluating  the  results  in  terms  of  their  prac¬ 
tical  implications  for  the  design  of  the  EMARS  system. 

■  . 

Methods  and  Results 

To  simulate  EMARS,  several  special  NETMAN  features  were 
used  and  appropriate  task  analyses  and  network  structures  were  devel¬ 
oped  : 


The  EMARS  concept  was  evaluated  by  parametrically  manipulat¬ 
ing  a  number  of  NETMAN  variables  and  assessing  the  impact  on  the  sys¬ 
tem's  performance  and  effectiveness.  The  variables  investigated  formed 
two  groups:  (1)  physical,  and  (2)  behavioral. 

The  effects  of  four  physical  variables  on  system  performance  were 
examined:  (1)  number  of  system  entry  messages  (field  situational  reports), 
(2)  number  of  messages  generated  per  entry  message  (each  report  may 
generate  more  than  one  entry  into  the  management,  control,  and  assess¬ 
ment  system),  (3)  length  of  the  system  entry  message  to  the  message  to 
processing  personnel,  and  (4)  message  length  to  staff  personnel. 

The  effects  on  EMARS'  performance  of  three  system  operator  or 
behavioral  variables  were  also  assessed:  (1)  proficiency,  (2)  precision, 
and  (3)  aspiration. 
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Twenty-two  computer  runs  were  completed  which  allowed  the  ef¬ 
fects  of  each  NETMAN  variable  which  was  manipulated  to  be  examined 
across  three  values  and  compared  with  a  baseline.  Whenever  possible, 
the  performance  and  effectiveness  of  the  EMARS  were  also  compared  with 
the  data  from  a  prior  NETMAN  application  (Siegel  et  al. ,  1979)  which 
simulated  a  TWSEAS-type  system  communication  net. 

The  findings  generally  suggested  support  for  the  EMARS  concept. 
EMARS  was  found  to  be  robust  up  to  a  message  frequency  of  10  per  hour 
per  operator--a  rather  high  message  frequency --and  field  message 
length  of  100  characters  per  message.  Within  the  parameters  examined, 
staff  message  length  had  little  effect  on  system  performance.  Also  ap¬ 
parent  was  the  rather  consistent  effect  of  operator  proficiency,  precision, 
and  aspiration  on  EMARS'  performance. 

In  comparison  to  the  TWSEAS-type  system,  the  EMARS'  perform¬ 
ance  and  effectiveness  were  observed  to  be  rather  consistently  superior. 
This  superiority  was  most  marked  when  a  large  number  of  messages  or 
lengthy  messages  are  involved. 

The  NETMAN  model  was  found  to  be  fully  adaptable  for  simulating 
the  EMARS  and,  by  implication,  the  utility  and  generality  of  NETMAN  for 
such  purposes  seems  supported. 

Implications 

The  findings  suggested  advantages  to  the  EMARS  concept  in  terms 
of  the  simulated  system's  message  processing  time,  throughput  capacity, 
and  a  number  of  measures  of  "effectiveness.  "  Accordingly,  it  seems  that 
the  EMARS  concept  is  viable  and  should  be  pursued. 

NETMAN'S  ability  t.u  simulate  the  EMARS'  concept  tends  to  sup¬ 
port  claims  favoring  its  adaptability.  This  contention  is  supported  by 
the  differential  nature  of  the  output  data  from  the  simulated  EMARS  and 
the  TWSEAS-type  system.  The  effectiveness  differences  between  sys¬ 
tems  were  orderly  and  could  be  meaningfully  related  to  real  differences 
in  network  structures. 

Taken  together  with  the  sensitivity  and  validity  testing  (Siegel  ct 
al. ,  1979),  the  effectiveness  and  utility  of  NETMAN  seems  to  have  been 
reasonably  demonstrated  and  NETMAN  may  be  considered  to  represent 
a  useful  tool  for  evaluating  message  processing  systems. 
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CHAPTER  I 


INTRODUCTION  AND  BACKGROUND 


Over  the  past  several  years,  the  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  has  supported  efforts  to  develop:  (1)  a 
stochastic,  behaviorally  oriented  computer  simulation  model  (NETMAN) 
for  simulating  message  throughput  in  Army  tactical  operations  systems, 
and  (2)  a  conceptual  design  of  an  Exercise  Monitoring  Assessment  and 
Reporting  System  (EMARS). 

Accordingly,  the  work  described  herein  possessed  two  interre¬ 
lated  purposes:  (1)  to  assess  the  merit  of  the  EMARS  concept  as  indi¬ 
cated  through  application  of  the  NETMAN  model,  and  (2)  to  evaluate  the 
"effectiveness"  or  "utility"  of  the  NETMAN  model  when  the  model  is  ap¬ 
plied  to  a  system  which  is  in  the  conceptual  phase  of  development. 

Prior  Developments 

The  basic  concepts  included  in  NETMAN  are  based  on  a  prior 
operational  computer  model  designed  by  Applied  Psychological  Services 
under  the  direction  of  the  U.S.  Army  Research  Institute  for  the  Behav¬ 
ioral  and  Social  Sciences.  The  earlier  model,  called  MANMOD,  simu¬ 
lates  the  U.S.  Army’s  Tactical  Operations  System  (TOS)--a  battlefield 
information  management  and  retrieval  system.  MANMOD  (Siegel,  Wolf, 
and  Leahy,  1973)  simulates  the  actions  of  up  to  six  men  functioning  as 
action  officers  and  of  input-output  device  operators  in  a  TOS. 

NETMAN's  predecessor,  MANMOD,  was  written  in  FORTRAN  for 
batch  run  processing  on  the  CDC  3300  computer.  Several  unique  aspects 
of  the  FORTRAN  available  on  this  system  were  used  in  the  MANMOD. 
MANMOD  was  extensively  tested  (Siegel,  Wolf,  and  Leahy,  1972). 

Subsequently,  MANMOD  was  expanded  to  operate  in  an  interactive 
computer  time  sharing  mode  (Siegel,  Wolf,  Leahy,  Bearde,  and  Baker, 
1973).  In  this  mode,  the  user  can;  (1)  enter  data  via  a  terminal,  (2) 
easily  control  changes  in  simulation  parameters,  and  (3)  assess  their  im¬ 
pact  on  output. 

Later  MANMOD  was  adapted  for  execution  on  the  Univac  1108 
computer.  The  system  specific  aspects  of  FORTRAN  were  removed  from 
MANMOD  and  several  simulation  features  were  added  to  increase  the 
overall  realism  of  the  simulation.  The  MANMOD  was  also  modified 


to  allow  the  exchange  of  data  with  two  independent  computer  models  in 
such  a  way  as  to  maximize  the  strong  points  of  each  computer  model 
(Leahy,  Siegel,  and  Wolf,  1975a,  1975b). 

The  simulation  technique  for  task-by-task  performance  evaluation 
in  MAN  MOD  was  adopted  in  the  development  of  NETMAN  (Siegel,  Leahy, 
and  Wolf,  1977). 

Other  features  from  MANMOD  used  in  NETMAN  include: 

•  Message  queues  are  generated  at  the 
beginning  of  each  simulation  iteration 

•  Operator  factors  (such  as  speed,  pre¬ 
cision,  level  of  aspiration,  and  stress 
threshold)  influence  performance 

•  Stochastic  variation  (i.  e. ,  random  com¬ 
binations  of  chance  events)  is  used  to  in¬ 
crease  the  generalizability  of  the  output 

•  Lists  of  inputs  and  run  summaries  are 
automatically  provided  with  various  ad¬ 
ditional  outputs  available  as  options. 

Overview  of  the  NETMAN  Model 


The  NETMAN  model  simulates  each  person  and  each  message  in¬ 
volved  in  the  data  acquisition  required  for  evaluating  performance  in 
field  exercise  control  and  monitoring  systems.  These  personnel  include 
up  to  27  referees,  27  radio  operators,  and  3  controllers  interacting  in  a 
fixed  network  of  communication  lines  linked  with  a  Central  Computing 
Center  (CCC). 

The  field  exercise  data  acquisition  situation  which  is  being  sim¬ 
ulated  may  be  viewed  as  a  message  processing  network  configured  as 
shown  in  Figure  1.  This  exhibit  symbolically  displays  27  simulated  refer¬ 
ees  (Ill  through  K27)  receiving  simulated  input  messages  from  independent 
sources  as  well  as  from  one  of  three  simulated  controllers  (CON  1,  CON  2, 
or  CON  3).  Messages  arc  indicated  by  the  symbol  C3  . 

Military  field  exercises  of  some  form  are  observed  by  referees, 
who  complete  evaluative  and  situational  reports  which  arc  transmitted  to 
a  computer  via  a  radio  operator.  Messages  introduced  into  the  system 
are  processed  by  various  personnel  and  the  CCC  and  then  delivered  to 
controllers  for  evaluation.  The  field  exercise  data  acquisition  is  simulat¬ 
ed  through  random  message  generation  based  on  pertinent  values  such  as 
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message  length,  type,  and  arrival  time.  Each  generated  message  is 
then  processed  through  the  referee  -*  radio  operator  -♦  CCC  -*  controller 
network,  and  processing  time  is  determined  along  with  a  number  of 
other  descriptive  indices. 

Each  simulated  referee  in  the  left  to  right  message  processing 
flow  of  Figure  1  performs  some  appropriate  procedure  on  the  simulated 
message(s)  received.  This  is  shown  symbolically  by  a  circle  in  which 
TAR  (Task  Analysis,  Referee)  appears.  A  message  then  passes  to  the 
corresponding  radio  operator  (one  of  ROj  through  RO27)  for  processing 
in  accordance  with  some  specified  task  analysis  procedure,  circle  TAO 
(Task  Analysis,  Operator). 

Up  to  2  7  simulated  messages,  each  processed  by  a  different  radio 
operator,  could  then  be  ready  for  entry  into  the  CCC.  Entry  into  the  sim¬ 
ulated  CCC  for  any  given  message  is  made  over  the  communication  line 
for  the  three  networks  shown.  Accordingly,  in  a  given  network,  there 
may  be  up  to  nine  simulated  messages  competing  for  the  one  available 
CCC  input  line. 

Telecommunication  lines  are  designated  in  Figure  1,  and  the  re¬ 
sultant  queues  awaiting  CCC  actions  are  designated  by  C3 . . .  £3, 

On  a  "first-in  first-out"  basis,  the  CCC  processes  messages  from 
all  of  the  three  networks  in  accordance  with  its  task  analysis  procedure -- 
depicted  by  circle  TAC  (Task  Analysis,  Computer).  These  messages 
then  enter  another  queue  awaiting  action  by  one  of  three  controllers.  Each 
simulated  controller  then  assesses  and  operates  on  the  oldest  message 
from  his  network  in  the  queue  and  performs  in  accordance  with  the  task 
analysis  for  controllers  as  symbolized  by  circle  TAN. 

The  loop  is  closed  by  the  simulated  generation  of  new  messages 
by  the  controller  for  input  to  one  of  the  referees  in  his  nine-way  network 
as  a  function  of  a  parameter  input  to  the  model.  Besides  the  link  from 
the  controller  to  his  referees,  the  referees  are  also  interconnected  and 
may  generate  new  messages  as  a  result  of  their  interactions  with  one 
another.  Up  to  5,  400  messages  may  be  included  in  a  given  simulation 
run.  During  NETMAN's  processing  of  message  information,  the  model 
places  special  emphasis  on  certain  human  performance  features  consid¬ 
ered  to  be  important  in  a  field  operational  system  of  this  sort.  These 
include  operator  stress,  fatigue,  level  of  aspiration,  proficiency,  and 
precision. 

The  overall  man-machine  performance  measure  calculated  by  the 
model,  effectiveness,  is  calculated  for  each  day  of  the  simulated  exer¬ 
cise.  It  is  composed  of  four  independent  factors --thor  oughness,  com¬ 
pleteness,  accuracy,  and  responsiveness. 
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Output  from  the  model  is  presented  in  the  form  of  computer  tabu¬ 
lations  and  terminal  displays.  These  provide  data  which  promote  insight 
for  evaluating  alternatives  both  in  terms  of  absolute  did  relative  value. 
The  printed  and  displayed  output  is  designed  and  organized  so  as  to  pro¬ 
vide  results  that  answer  questions  of  practical  importance,  such  as: 

1.  What  is  the  average  processing  time  for  a 
message  which  passes  through  the  network 
as  a  function  of  the  frequency  of  input  mes¬ 
sages,  operator  capabilities,  and  network 
configuration? 

2.  How  do  changes  of  input  parameters  affect 
predicted  total  man- machine  effectiveness 
values? 

3.  What  is  the  loading  situation  relative  to  idle 
vs.  busy  time  for  referees,  radio  operators, 
the  CCC,  and  the  controller? 

4.  How  great  a  stress  is  placed  on  the  operators? 

What  is  the  fatigue  profile  over  the  mission  for 
each  type  of  personnel? 

5.  Would  changes  in  the  task  organization  of  one 
or  more  operators  materially  affect  average 
processing  time  and  system  effectiveness? 

6.  What  are  some  effects  of  operator  commission 
and  omission  errors  under  various  conditions? 

7.  How  would  increased  personnel  training  or  im¬ 
proved  personnel  selection  affect  system  per¬ 
formance? 

The  program  presents  detailed  message  processing  time  and  er¬ 
ror  information,  if  desired,  as  well  as  hourly  summary  and  run  sum¬ 
mary  outputs.  The  detailed  message  processing  output  shows  the  fine 
grain  of  the  results  of  the  simulation  of  each  task  in  the  processing  of 
messages. 

The  hourly  summary  presents  a  consolidation  of  the  results  of  a 
simulated  hour's  work  across  all  iterations  and  includes  items  such  as; 
number  of  messages  completed,  time  spent  working,  end  of  hour  stress 
level,  performance  and  aspiration,  time  spent  performing  various  proc¬ 
esses,  and  average  time  per  message. 
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The  simulation  run  summary,  produced  after  N  iterations  of  the 
exercise,  includes  manpower  utilization,  message  processing  time, 
overall  effectiveness  indicators,  and  workload  summary  information. 

NETMAN  is  programmed  in  FORTRAN  IV  for  the  Univac  1108 
system.  It  is  organized  to  allow  the  user  to  conduct  various  numerical 
"experiments."  Each  computer  run  of  the  model  represents  a  simulation 
of  a  field  session  up  to  10  hours  in  duration  conducted  under  conditions 
as  specified  by  input  parameters.  Examples  of  exercise  input  param¬ 
eters  include  the  frequency  of  messages  entered  to  the  system,  the  num¬ 
ber  of  operators,  and  the  speed  and  aspiration  levels  of  these  operators. 

A  full  model  description  is  contained  in  Siegel,  Leahy,  and  Wolf 
(1977)  together  with  a  discussion  on  model  utilization,  program  flow 
charts,  subroutine  definitions,  user  input-output  formats,  and  task  anal¬ 
yses.  Recent  revisions  to  the  model  are  described  in  the  updated  user's 
manual  (Leahy,  Siegel,  and  Wolf,  1980)  and  a  full  historical  summary 
has  been  prepared  (Siegel  and  Wolf,  in  press). 

The  EMARS  Concept 

The  EMARS,  one  of  the  major  concerns  of  this  report,  is  con¬ 
ceived  (Siegel  et  al. ,  1980)  as  a  computer  assisted,  exercise  command 
and  control  system  with  capability  to  support  field  exercises  conducted 
by  the  Army  up  to  and  including  those  at  the  Company  level.  While  de¬ 
ployed  operationally,  each  EMARS  system  is  outfitted  for  sustained  op¬ 
eration  at  Division  level  headquarters  in  which  it  supports  successive 
Army  combat  unit  exercises.  EMARS  fully  exploits  the  potential  of  com¬ 
puter  assisted  tactical  exercise  monitoring  as  a  training  and  testing 
medium. 

The  overall  EMARS  functions  include: 

1.  accepting  and  processing  digital  data  and 
information 

2.  generating  situation  displays 

3.  assessing  performance  and  generating 
objective  scores  for  individual  and  com¬ 
bined  military  units  at  all  tested  levels 
by  time  period 

4.  supporting  exercises  up  to  four  days  in 
duration 
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5.  providing  multiple  hard  copy  results- - 
maps,  records,  scores,  and  com¬ 
ments. 


Overall  System  Description 

The  EMARS  concept  includes  a  system  which  is  contained  in  a 
single  van  and  is  fully  transportable.  Figure  2  presents  an  external  con¬ 
ception  of  the  system  and  Figure  3  presents  a  preliminary  conception  of 
the  internal  arrangement  of  the  persons  and  equipments  in  the  system. 

The  personnel  complement,  other  than  system  maintenance  per¬ 
sonnel,  consists  of  five  persons:  personnel/ logistics  monitor  (PER/LOG), 
operations/ intelligence  monitor  (OPS/INT),  indirect  fire/air  space  man¬ 
ager  (IF/ ASM),  exercise  monitoring  officer  in  charge  (EXMOIC),  and  ex¬ 
ercise  management  information  officer  in  charge  (EMIC). 

All  information  processing  and  exercise  monitoring  functions  are 
performed  by  these  persons  who  are  tied  by  communication  lines  to  ex¬ 
ercise  control  personnel  at  Division.  The  first  three  EMARS'  personnel, 
PER/ LOG,  OPS/INT,  and  IF/ ASM  receive  situation  information,  verify 
it,  and  enter  the  information  into  the  computer  system  for  processing 
and  storage.  The  other  personnel,  the  EXMOIC  and  the  EMIC  possess 
the  capability  to  call  information  from  the  system  at  any  point  in  time 
and  to  provide  a  wide  variety  of  status  and  performance  information  for 
each  unit  involved  in  an  exercise.  All  of  the  personnel  possess  access 
to  the  performance  data  base  and  each  possesses  an  interactive  terminal 
with  a  cathode  ray  tube. 

A  hard  copy  plotter  is  provided  to  allow  overall  presentation  of 
unit  disposition,  the  planned  and  actual  progress  of  each  unit,  a  time 
history  of  the  progress  of  each  unit,  and  significant  events  affecting 
progress.  Hard  copy  facsimile  can  be  produced  on  any  cathode  ray 
tube  presentation  available  to  the  operating  personnel.  These  displays, 
their  formats,  frequency  of  use,  criticality,  and  the  anticipated  user(s) 
were  described  previously  (Siegel  et  al. ,  1980).  In  all,  the  system  pos¬ 
sesses  the  capability  for  automatically  producing  23  different  types  of 
status  or  performance  reports.  These  individual  reports  also  form  the 
basis  for  ar  "after  action  review.  " 

Application  of  NETMAN  to  EMARS 

The  NETMAN  model  possesses  several  characteristics  which  al¬ 
low  it  to  be  useful  in  testing  exercise  monitoring  and  control  systems. 
First,  NETMAN  is  based  on  the  logic  that  the  basic  unit  of  measurement 
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Figure  2.  External  view  of  EMARS 


HARO  COPY  PLOTTER 


Figure  3.  EMARS'  internal  configuration. 


in  any  communication  net  is  the  message,  its  processing  time,  and  its 
transmission  accuracy.  Related  to  these  concerns  are  NETMAN's  abil¬ 
ity  to  allow  simulation  of  the  effects  of  varying  message  characteristics 
such  as  message  frequency  and  length. 

Second,  the  NETMAN  model  allows  the  capability  to  assess  and  to 
simulate  differentially  the  effects  of  a  number  of  important  operator  var¬ 
iables  on  system  performance.  This  simulation  and  assessment  may  be 
achieved  very  systematically.  In  NETMAN,  the  performance  of  each  op¬ 
erator  is  a  function  of  psychological,  behavioral,  and  physiological  var¬ 
iables.  Operator  speed,  precision,  aspiration,  fatigue,  and  stress  tol¬ 
erance  all  can  be  specified  and  varied  to:  (1)  assess  their  effects  on  total 
system  throughput,  and  (2)  enable  a  simulation  to  reflect  realistically 
what  can  be  expected  in  the  operation  of  the  system. 

Third,  different  exercise  monitoring  and  control  systems  impose 
different  behaviorial  repertories  and  task  sequences  on  their  operators. 
These  can  be  differentially  represented  in  the  NETMAN  model.  This 
flexibility  of  the  NETMAN  model  has  been  clearly  demonstrated  in  ear¬ 
lier  simulations  employing  NETMAN.  These  involved  the  Army's  TOS 
system  (Siegel  et  al. ,  1972)  and  the  Marine  Corps'  Tactical  Warfare 
Simulation  Analysis  and  Evaluation  System  (TWSEAS)  (Siegel  et  al. , 
1979). 


Content  of  Subsequent  Chapters 

Chapter  II  of  this  report  presents  the  details  of  applying  the  NET- 
MAN  model  to  the  EMARS  system  for  the  dual  purposes  on  hand.  The  re¬ 
sults  are  presented  in  Chapter  III  and  their  relationship  to  the  goals  of  the 
work  is  discussed  in  Chapter  IV. 
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CHAPTER  II 


METHOD  OF  APPLICATION  OF  NETMAN  TO  EMARS 


The  effectiveness  of  the  EMARS  concept  was  evaluated  by  system¬ 
atically  manipulating  a  number  of  NETMAN  variables.  The  variables 
served  the  role  or  function  assigned  to  independent  variables  in  a  formal 
experiment.  That  is,  each  variable  was  directly  and  independently  ma¬ 
nipulated  over  several  levels.  Each  selected  variable  was  independently 
manipulated  across  four  values  and  the  effects  of  the  manipulation  on  mod¬ 
el  output  were  assessed. 


Structure  of  Simulation 


The  EMARS  concept  includes  a  five  person  crew:  personnel/lo- 
gistics  monitor  (PER/ LOG),  operations/ intelligence  monitor  (OPS/INT), 
indirect  fire/air  space  manager  (IF/ ASM),  exercise  management  and  in 
formation  controller  (EMIC),  and  exercise  monitoring  officer  in  charge 
(EXMOIC).  The  first  three  of  these  (subsequently  called  message  proc¬ 
essing  personnel)  are  conceived  to  possess  the  responsibility  for  receiv¬ 
ing  and  assembling  information  relative  to  their  respective  interest  areas, 
coding  the  information,  and  entering  the  information  into  the  data  manage¬ 
ment  system.  The  final  two  EMARS  personnel  (subsequently  called  moni¬ 
toring  personnel),  the  EMIC  and  the  EXMOIC,  possess  decision  making 
responsibility. 

Accordingly,  the  PER/ LOG,  OPS/INT,  and  IF/ ASM  personnel 
were  conceived  as  analogous  to  the  radio  operators  in  the  NETMAN 
scheme  and  the  EMIC  and  the  EXMOIC  were  conceived  as  controllers  in 
the  NETMAN  scheme. 

Referees  are  not  included  in  the  EMARS  concept.  Accordingly, 
the  referee  representation  in  NETMAN  was  not  relevant  to  the  present 
simulation. 

Some  control  systems  may  be  designed  so  that  the  functions  of  one 
or  more  positions  (levels)  are  automated  or  otherwise  replaced.  To  in¬ 
crease  the  generality  of  NETMAN  and  its  applicability  to  such  systems, 
the  capability  to  bypass  a  system  level  during  message  processing  was 
recently  added  to  the  NETMAN.  Advantage  of  this  feature  was  taken  dur¬ 
ing  the  EMARS  simulation.  The  system  level  bypass  feature  was  imple¬ 
mented  to  bypass  the  referee  level  of  the  NETMAN  model  although  mes¬ 
sage  generation  was  simulated.  The  referee  level  was  bypassed  in  the 
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simulation  because  the  EMAHS  concept  assumes  field  data  to  be  acquired 
through  automatic  methods  or  that  field  information  is  delivered  manually 
to  the  EMAHS  van.  EMAHS  does  not  include  referees.  In  EMAHS,  once 
the  information  arrives  at  the  van,  it  is  prepared  (message  generation) 
for  entry  into  the  computer  based  system.  The  result  was  a  more  ver¬ 
idical  representation  of  EMAHS  in  NETMAN. 


Accordingly,  the  EMAHS  simulation  involved  three  NETMAN  lev¬ 
els,  The  assumed  analogies  between  NETMAN  and  EMAHS  are; 


CJ 

NETMAN 

EMAHS 

Position(s) 

HO 

PEH/LOG,  OPS/1NT,  IF/ ASM, 

Controller 

EXMOIC,  EM1C 

Processor 

Computer 

Computer 

This  organization  allowed  full  simulation  of  the  EMAHS  including 
receipt  of  field  based  information  by  EMAHS'  message  processing  per¬ 
sonnel,  (PEH/LOG,  OPS/ IN  T,  and  IF/ ASM),  entry  of  these  data  by  EMAHS' 
personnel  into  a  computer  based  management  system  and  processing  of 
these  data  by  the  computer,  and  action  on  the  basis  of  the  data  by  EMAHS' 
monitoring  personnel  (EM1C  and  EXMOIC). 

Stated  alternatively,  the  EMAHS'  personnel,  their  functions,  and 
their  numbers,  were  defined  in  NETMAN  as  two  operating  networks.  In 
each  network,  three  levels  were  represented;  (1)  PEH/LOG,  OPS/ IN T, 
and  1 1*7 ASM,  (2)  computer  and  its  systems,  and  (3)  EM1C  and  EXMOIC. 

Figure  4  presents  the  representation  of  each  of  the  EMAHS  system 
within  the  general  NETMAN  flow. 

Task  Analyses 

The  NETMAN  model  requires  as  input  a  task  analysis  which  re¬ 
flects  the  lask  elements  required  for  the  performance  of  each  task  of  each 
of  the  simulated  personnel. 

Task  analytic  input  data  for  each  of  the  simulated  EMAHS'  posi¬ 
tions  are  presented  in  Tables  1  and  2.  Figures  5  and  6  present  the  flow 
of  the  task  analysis  for  each  position. 

In  Tables  1  and  2,  each  task  element  is  operationalized  by  its  mean 
time  in  seconds  for  completion,  the  standard  deviation  of  this  time,  and  the 
probability  that  the  element  will  be  successfully  performed. 
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Table  1 


Task  Analysis  for  PER/ LOG, 
OPS/INT,  and  IF/ ASM  EMARS'  Personnel 


Time  Next  Element  If 


(seconds) 

Pi'esent  Element 

Task 

Elements 

Mean 

S.D. 

Success 

Probability 

Succeeds 

Fails 

1.  Receive 
and  tran- 

scribe 

80.  00 

30.00 

.98 

2 

1 

2.  Check, 
edit,  and 

query 

30.00 

12.00 

.  98 

3 

2 

3.  Code 

15.00 

3.00 

.  98 

4 

3 

4.  Type 

85.00 

37.00 

.  98 

5 

4 

5.  Verify 

20.00 

12.00 

.98 

6 

3 

G.  End 

0. 

0. 

1.0 
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Table  2 


Task 

Elements 

1.  Observe 
CRT  mes¬ 
sage 

2.  Decide 

3.  Query 
(Type) 

4.  Observe/ 
decide 

5.  Get  hard 
copy 

6.  Query 
(Type) 

7.  Observe 

8.  Talk 

9.  End 


Task  Analysis  for 

EMIC  and  EXMOIC  EMARS'  Personnel 


Time 

Next  Element  If 

(seconds) 

Success 

Present  Element 

Mean 

S.  D. 

Probability 

Succeeds 

Fails 

0.  10 

0.  10 

.  99 

2 

1 

5.  00 

2.  00 

.75 

3 

9 

3.  00 

2.  00 

.  99 

4 

3 

20.  00 

10.00 

.75 

5 

6 

6.  00 

0.  50 

.  99 

6 

5 

3.  00 

2.  00 

.  99 

7 

6 

20.  00 

10.  00 

.  75 

8 

9 

160. 00 

60.  00 

.  99 

9 

8 

0.  01 

0.  01 

1.  0 
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Figure  5.  Work  flow  for  PER/ LOG,  OPS/IXT,  and  IF/ ASM  EMARS'  personnel. 


for  EMIC  and  EXMOIC  EMARS'  personnel. 


The  task  element  times  were  largely  derived  from  measure¬ 
ments  of  the  time  to  complete  such  task  elements  on  current  systems. 

The  data  are  believed  to  represent  best  available  estimates. 

The  first  two  task  element  times  for  the  PER/ LOG,  OPS/1NT, 
and  the  IF/ ASM's  task  analysis  (Table  1)  were  derived  by  partitioning 
the  measured  time  for  TWSEAS'  radio  operators  to  transmit  a  message 
(except  in  EMAliS,  the  personnel  received  the  message).  The  last  three 
task  element  times  were  derived  through  a  set  of  measurements  of  the 
time  required  by  a  program  staff  member  to  code,  type,  and  verify  a 
number  of  TWSEAS  messages. 

The  task  element  time  assignments  for  the  EM1C  and  the  EXMOIC 
(Table  2)  were  derived  from  stop  watch  measurements  of  the  time  requir¬ 
ed  by  a  program  staff  member  to  perform  each  of  the  task  elements  at  an 
in-house  cathode  ray  tube  terminal.  Task  elements  1  and  2  for  the  EM1C 
and  the  EXMOIC  were  the  time  required,  on  the  average,  to  read  a  CRT 
alerting  message  and  thoughtfully  decide  on  a  course  of  action.  The  third 
and  the  sixth  task  clement  times  represented  the  mean  time  and  its  stand¬ 
ard  deviation  to  type  a  two  or  three  word  command  into  a  computer.  The 
time  for  the  fourth  and  seventh  task  elements  of  the  EMJC  and  EXMOIC 
were  based  on  estimates  of  the  time  to  study  a  chart  or  a  graph  and  to  de¬ 
rive  relevant  information.  The  time  for  task  element  5  of  the  personnel 
at  this  system  level  was  the  measured  time  for  a  conventional  photocopier 
to  produce  a  copy.  Finally,  tile  time  for  task  clement  8  was  the  mean  total 
time  (and  its  standard  deviation)  required  by  TWSEAS'  referees  to  compose 
a  message. 

Task  Flow 


The  simulated  task  element  flow  for  the  PER/LOG,  OPS/1NT  and 
IF/ ASM  personnel  of  the  EMARS  concept  is  presented  in  Figure  5.  The 
first  task  element  represents  the  time  to  receive  and  transcribe  a  mes¬ 
sage.  The  second  element  embodies  the  operators'  performance  in  check¬ 
ing  and  editing  the  transcribed  message  and  making  queries  about  its  con¬ 
tents,  intent,  or  classification.  The  next  element  represents  the  behav¬ 
iors  necessary  to  code  tile  message  for  input  into  the  computer.  Task 
element  4  indicates  the  time  necessary  to  enter  the  appropriately  coded 
message  into  the  computer.  This  requires  specifying  the  nature  of  the 
data  and  typing  it  into  the  computer.  The  final  element  expresses  the  time 
needed  to  verify  thoughtfully  the  information  entered  and  for  storing  the 
information  in  the  data  base.  The  act  of  storing  the  data  was  assumed  to 
generate  an  alerting  message  on  the  CRT  of  the  EM1C  or  EXMOIC. 

The  probability  of  successfully  performing  the  first  four  task  ele¬ 
ments  was  set  at  0.  08.  For  these  four  task  elements,  two  percent  of  the 
time,  an  error  was  assumed  to  occur,  to  be  detected,  and  to  be  corrected 
by  the  operator  who  repeats  the  necessary  element.  The  possibility  was 
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assumed  that  a  conceptual  error  in  coding  would  be  made  by  the  opera¬ 
tor  and  Dial  the  error  would  escape  detection  umil  the  final  verification 
element.  Therefore,  an  error  detected  during  the  performance  of  the 
fifth  element  required  return  to  the  third  elcment--the  coding  element. 

This  means  that  the  third,  fourth,  and  fifth  elements  needed  to  be  repeat¬ 
ed  by  the  simulated  operator. 

The  structuring  and  dynamics  of  the  task  analysis  for  the  moni¬ 
tors,  the  EMiC  and  the  EXMOJC,  are  presented  in  Figure  G.  The  first 
element  is  indicative  of  the  time  to  read  a  one  or  a  two  word  alerting 
message  presented  on  a  CUT.  The  message  is  intended  to  alert  the  mon¬ 
itor  of  a  change  in  the  data  base.  After  the  reading  of  the  alerting  mes¬ 
sage,  the  second  task  element  represents  the  time  required  to  decide 
whether  or  not  the  alert  should  bo  follow  ed  up;  that  is,  should  the  new  in¬ 
formation  in  the  data  base  be  examined.  If  the  simulated  operator  decides 
to  examine  the  data  base,  the  task  flow  proceeds  to  the  next  element. 
However,  if  the  simulated  operator  decides  not  to  examine  the  change(s) 
in  the  data  base,  the  task  flow  is  stopped.  The  third  task  element  requires 
the  monitor  to  solicit  the  information  in  the  changed  data  base  while  the 
fourth  element  represents  the  examination  and  consideration  of  this  infor¬ 
mation  by  the  simulated  monitor.  Also  represented  in  this  element  is  the 
choice  between  obtaining  a  hard  copy  or  not.  If  the  monitor  decides  on 
hard  copy,  the  task  flow  proceeds  to  task  element  5.  If  not,  the  flow  pro¬ 
ceeds  to  element  6.  Task  element  G  is  another  query  element  in  which  ad¬ 
ditional  information  is  accessed.  The  additional  information  is  studied  as 
element  7.  Task  element  7  requires  a  decision  on  whether  to  take  an  ac¬ 
tion  or  to  end  the  task  flow.  The  action,  embodied  in  element  8,  is  either 
to  contact  a  field  observer  or  to  write  a  memo  for  future  use.  When  either 
of  the  two  actions  is  not  performed,  task  element  8  is  skipped  and  the  task 
flow'  is  terminated  at  element  9. 


Parametrically  Treated  Variables 


Seven  NETMAN  input  variables  were  selected  for  parametric  ma¬ 
nipulation  in  the  present  work- -four  physical  and  three  operator  parame¬ 
ters.  These  parameters,  along  with  their  variation  over  model  runs  in 
the  EM AUS  simulation,  are  presented  in  Table  3. 

The  first  two  physical  factors  were  concerned  with  the  number  of 
messages  involved.  The  number  of  messages,  of  course,  determines  the 
load  on  the  system  and  the  stress  on  the  simulated  personnel.  The  next 
two  physical  factoi were  concerned  with  the  length  of  the  messages;  (1) 
received  by  the;  processing  personnel,  and  (2)  received  by  the  monitors. 
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The  operator  parameters  were  concerned  with  the  proficiency  of 
the  simulated  assigned  personnel,  their  precision,  and  their  level  of  as¬ 
piration. 

There  were  22  operational  computer  simulation  runs  and  a  base¬ 
line  run.  The  operational  runs  involved  a  wide  spread  of  variations  with¬ 
in  each  parameter.  In  each  run,  only  one  parameter  was  varied  while  all 
others  were  held  at  baseline  value.  The  baseline  value  of  each  variable 
is  indicated  in  a  separate  column  of  Table  3.  These  values  were  essen¬ 
tially  "normal"  operating  conditions  whose  output  was  compared  with  the 
output  produced  by  the  other  parametric  variations  listed. 

The  "control"  column  of  Table  3  represents  the  input  to  a  previ¬ 
ously  completed  simulation  run  for  the  TWSEAS  system  (Siegel,  Leahy, 
and  Wolf,  197  9).  The  run  selected  is  the  available  prior  run  which  most 
closely  approaches  the  baseline  parameters  of  the  present  simulations. 
The  representation  of  the  TWSEAS  system  in  NET MAN  is  shown  in  Figure 
7. 


Each  of  the  runs  was  essentially  a  replication  of  the  baseline  run 
with  one  parameter  varied.  All  the  input  values  listed  under  the  " Base¬ 
line"  heading  of  Table  3  were  included  with  the  exception  that  the  numeri¬ 
cal  value  specified  under  the  appropriate  run  number  was  substituted  fox- 
the  corresponding  baseline  value.  For  example,  in  run  1  the  number  of 
messages  generated  per  stimulus  message  was  set  to  two- -the  two  was 
substituted  for  the  baseline  value  of  one  message  generated  per  stimulus 
message.  Likewise  in  run  4,  the  ten  was  substituted  for  the  mean  of 
three  messages  per  hour  in  the  baseline  and  four  was  substituted  for  the 
standard  deviation  of  two  messages  per  hour. 

Each  run  simulated  system  performance  over  a  four  hour  period 
and  was  based  on  five  iterations.  This  number  of  iterations  has  been 
found  previously  to  yield  a  stable  result. 
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CHAPTER  III 


RESULTS  AND  DISCUSSION 


The  parametric  variations  described  earlier  led  to  a  variety  of 
data  which  were  employed  to  evaluate  both  the  EMARS  system  and  the 
utility  of  the  NETMAN  model  for  such  system  evaluative  purposes. 


Message  Completion 


The  number  of  messages  moving  through  the  simulated  networks 
was  manipulated  by  varying  two  NETMAN  input  variables:  (1)  the  number 
of  messages  generated  per  stimulus  messages,  and  (2)  the  number  of 
stimulus  messages  per  processing  person  per  hour  (see  Table  3).  Func¬ 
tionally,  both  variables  had  the  same  effect- -either  to  augment  or  attenu¬ 
ate  message  frequency. 

The  number  of  simulated  messages  generated  per  stimulus  mes¬ 
sage  was  either  1  (baseline),  2.  3,  or  4  (runs  1,  2,  and  3  of  Table  3)  and 
the  mean  message  frequency  was  either  3  (baseline),  10,  20,  or  30  per 
hour  (runs  4,  5,  and  6  of  Table  3).  This  translates  to  a  message  frequency, 
on  the  average,  of  3  (baseline),  6,  9,  10,  12,  20,  or  30  per  operator  per 
hour.  The  results  of  these  simulations  differentiated  on  the  basis  of  op¬ 
erator  type  (PER/ LOG,  OPS/INT,  and  IF/ ASM  combined  or  EXMOIC  and 
EMIC  combined)  are  presented  in  Figure  8  in  which  the  number  of  mes¬ 
sages  completed  is  plotted  as  a  function  of  the  mean  message  frequency. 

The  effects  of  increasing  the  mean  message  frequency  were  to  in¬ 
crease  substantially  the  number  of  messages  completed.  The  rate  of 
change  was  not  constant  across  the  rather  large  increase  in  message  fre¬ 
quency.  Rather,  the  distribution  of  the  messages  completed  for  both  the 
processing  personnel  (PER/ LOG,  OPS/INT,  and  IF/ ASM)  and  the  moni¬ 
tors  (EXMOIC  and  EMIC)  tended  to  be  ogival.  A  positive  rate  of  change 
was  observed  in  the  number  of  messages  completed  up  to  a  mean  message 
frequency  of  about  10  per  hour  per  operator.  Message  frequencies  above 
10  tended  to  be  associated  with  increasingly  smaller  changes  in  the  rate 
of  messages  completed.  This  suggests  an  upper  level  for  the  EMARS  sys¬ 
tem  of  about  10  messages  per  operator  per  hour. 

Comparing  the  messages  completed  across  personnel  levels  indi¬ 
cated  an  increasingly  large  difference  as  message  frequency  increased. 
These  differences  were  probably  due  to  the  fact  that  the  two  simulated 
communication  nets  had  different  simulated  work  loads.  The  first  net. 
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Figure  8.  Number  of  messages  completed  as  o  function  of 
message  frequency  per  hour  per  operator. 


consisted  of  three  persons  (PER/ LOG,  OPS/INT,  and  EXMOIC)  while 
the  second  net  was  composed  of  two  persons  (IF/ASM  and  EMIC).  Ac¬ 
cordingly,  the  simulated  monitor  in  the  second  net  had  to  process  more 
output  to  keep  up  with  the  processing  personnel  at  the  first  level.  This 
finding  suggests  thatEMARS'  function  might  be  improved  by  adding  a 
monitor . 

However,  regardless  of  the  differences  between  levels,  the  most 
relevant  finding  concerned  the  increasingly  large  number  of  messages 
completed.  The  positive  relationship  observed  between  messages  com¬ 
pleted  and  message  frequency  was  apparently  achie /ed  because  of  the 
ability  of  the  simulated  E MARS'  personnel  to  adjust  to  the  changes  in  de¬ 
mand.  The  message  processing  was  accelerated  by  two  related  operator 
adjustments:  (l)the  time  per  message  was  reduccd--the  operators  work¬ 
ed  faster,  and  (2)  the  time  spent  working  increased.  These  adjustments 
are  reflected  in  Figure  9  in  which  time  per  message  and  percentage  time 
busy  are  plotted  as  a  function  of  message  frequency. 
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The  time  per  message  declined  [Figure  9  (A)]  with  increases  in  the 
message  frequency.  This  decline  was  most  apparent  for  the  processing 
personnel.  For  these  persons,  the  processing  time  per  message  de¬ 
creased  by  approximately  twenty  percent  as  message  frequency  increased 
from  3  to  30  per  operator  per  hour. 

Less  decline  was  found  in  the  monitors'  processing  time  per  mes¬ 
sage.  Here,  the  decline  was  only  about  six  percent  of  the  maximum.  This 
lesser  time  decline  may  be  due  to  the  unequal  work  load  between  the  two 
levels.  The  monitors,  possessing  a  heavier  work  load,  may  have  been 
forced  to  speed  up  earlier  to  their  maximum  rate  and  to  level  off  at  this 
maximum. 

The  results  for  the  other  NETMAN  output  (percentage  time  busy), 
which  reflected  the  simulated  behavior  of  the  E MARS'  personnel,  are  pre¬ 
sented  in  Figure  9(B).  The  distributions  of  the  busy  time  were  similar  to 
the  distributions  of  the  messages  completed.  That  is,  they  were  somewhat 
positively  accelerated  up  to  a  frequency  of  1  0  messages  per  hour  and  neg¬ 
atively  accelerated  thereafter.  Apparently,  the  increased  message  load 
was  met  by  the  simulated  operators  working  more  and  working  faster. 

Stress 


Concomitant  with  the  two  adjustments  in  simulated  performance 
were  concurrent  and  interdependent  changes  in  the  mean  stress  of  the  sim¬ 
ulated  personnel.  Succinctly,  NETMAN  operates  such  that,  as  percentage 
time  busy  increases,  stress  on  the  simulated  operators  increases;  as 
stress  increases  the  pace  of  the  simulated  operators  increases  and  the 
time  for-  message  completion  decreases.  The  mean  stress,  differenti¬ 
ated  on  the  basis  of  type  of  operator,  is  plotted  as  a  function  of  message 
frequency  in  Figure  10.  The  distributions  of  stress  tended  to  be  flat  un¬ 
til  a  moderate  message  frequency  value  was  reached.  At  this  point, 
mean  stress  accelerated  over  the  remaining  message  frequencies.  The 
value  at  which  the  stress  level  changed  varied  across  operator  levels. 
The  processing  personnel's  stress  was  invariant  up  to  a  message  frequen¬ 
cy  of  nine.  Beyond  that  value,  stress  increased  at  a  decreasing  rate  and 
reached  1.90  at  the  highest  message  frequency  of  30.  The  monitors’  mean 
stress  was  constant  up  to  10  messages  per  hour  per  operator.  Then,  the 
stress  level  increased  and  reached  a  value  of  1.70  at  a  message  frequen¬ 
cy  of  30.  In  neither  case  did  the  mean  stress  appear  to  asymptote. 

Accordingly,  it  seems  that  the  operators  in  the  EMARR  system 
are  placed  under  moderate  stress  under  higher  message  frequencies  and 
that  methods  may  be  needed  to  ameliorate  their  stress. 
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Figure  10. Mean  stress  as  a  function  of  message  frequency. 


Overall  Kffcctivcness 

The  NIST  MAN  model  also  provides  a  measure  of  overall  effective¬ 
ness.  The  overall  effectiveness  index,  with  a  range  of  0.0  to  1.0,  repre¬ 
sents  a  weighted  geometric  mean  of  four  component  measures;  (1)  thor¬ 
oughness,  (2)  completeness,  (3)  responsiveness,  and  (4)  accuracy.  The 
thoroughness  component  represents  the  r  atio  of  messages  completed  to 
the  total  messages  available  to  be  started.  Completeness  is  related  to 
the  success/ failure  ratio  for  task  elements  across  personnel  and  mes¬ 
sages;  it  can  be  thought  of  as  the  average  performance  level.  Responsive¬ 
ness  is  determined  as  the  message  processing  time,  which  includes  man 
and  computer  working  time,  divided  by  the  total  time  to  process  the  mes¬ 
sage,  which  also  includes  delays  and  time  spent  waiting  in  queue.  Accu¬ 
racy  is  a  function  of  information  Loss  within  the  system. 

Overall  effectiveness  is  plotted  against  message  frequency  per  op¬ 
erator  per  hour  in  Figure  11.  Over  the  message  frequency  range  involved, 
effectiveness  varied  about  eight  percent  with  a  range  of  0.83  to  0.94.  The 
shallowness  of  the  overall  effectiveness  decline  suggests  that  KMAliS  is 
reasonably  suited  to  a  task  of  handling  increasingly  greater  quantities  of 
incoming  messages  and  moving  them  through  the  system.  This  observa¬ 
tion  is  supported  through  a  comparison  of  the  calculated  effectiveness  of 
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Figure  11.  Overall  effectiveness  for  EM  ARS  and  TWSEAS- 
type  systems  as  a  function  of  message  frequency. 


the  EMARS  system  with  the  results  from  the  NETMAN  sensitivity  testing 
which  was  based  on  a  TWSEAS-typc  system  (Siegel  et  al. ,  1979).  As  in¬ 
dicated  in  Figure  11,  the  overall  effectiveness  of  the  TWSEAS-typc  system 
was  not  only  considerably  lower  than  EMARS  but  also  the  overall  effec¬ 
tiveness  of  the  TWSEAS-type  system  fell  to  about  0.63  when  the  message 
frequency  reached  30.  The  corresponding  overall  effectiveness  for  EMARS 
was  0.86. 

Thoroughness 


Examination  of  the  components  of  the  overall  effectiveness  index 
indicated  that  the  declining  relationship  was  almost  exclusively  a  function 
of  the  thoroughness  component.  The  thoroughness  index  for  EMARS  de¬ 
clined  from  0.93  to  0.67  as  message  frequency  increased  from  3  to  30 
per  hour  per  operator. 
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In  the  NETMAN  sensitivity  testing  with  a  TWSEAS-type  system, 
changes  in  message  frequency  over  a  range;  of  5  to  30  (a  range  compara¬ 
ble  to  the  3  to  30  range  used  in  the  EMAKS'  evaluation)  produced  a  very 
strong  reduction  in  thoroughness.  With  message  frequencies  of  30,  the 
thoroughness  measure  for  the  TWSEAS-type  system  was  between  0.  10 
and  0.20.  This  was  very  much  below  the  0.07  measure  reported  for 
EMAKS. 

Figure  12  presents  the  thoroughness  index  for  both  systems  plotted 
against  message  frequency.  When  message  frequency  was  low,  the  thor¬ 
oughness  indices  for  both  the  EMAKS  and  TWSEAS-type  systems  were 
similar.  As  message  frequency  increased,  the  thoroughness  for  both  sys¬ 
tems  declined.  However,  there  was  an  increasingly  large  difference  be¬ 
tween  systems.  It  appears  that  the  TWSEAS-type  system  degraded  much 
more  rapidly  and  to  much  lower  level  with  increases  in  message  frequenc  y 
than  the  EMAKS  system. 


Field  Message  Length 

The  next  set  of  variable  manipulations  examined  the  length  of  the 
messages  received  by  the  processing  personnel.  The  effects  of  message 
lengths  of  10,  30,  100,  and  1000  characters  per  message  were  evaluated. 

The  relationship  between  field  message  length  and  the  number  of 
messages  completed  is  presented  in  Figure  13.  The  number  of  messages 
completed  appeared  to  be  rather  independent  of  field  message  length  up  to 
a  length  of  1000  characters.  The  number  of  messages  completed  was  be¬ 
tween  240  and  2(10  for  both  the  processing  personnel  and  the  monitors  when 
the  field  message  length  was  10,  30,  or  100  characters  per  message.  How¬ 
ever,  when  the  length  was  1000  characters,  the  messages  completed  dropped 
to  about  20  for  all  of  the  EMAKS  personnel.  This  stability  of  output  was 
apparently  achieved  by  the  concomitant  adjustments  in  the  message  proc¬ 
essing  time,  as  affected  by  the  stress  variable  within  the  NETMAN  mod¬ 
el's  logic.  However,  EMAKS  seems  to  be  message  length  sensitive  and 
limiting  the  number  of  characters  in  any  message  seems  indicated  for  the 
system. 

The  time  per  message  is  plotted  as  a  function  of  the  field  message 
length  in  Figure  14.  The  effects  of  field  message  length  on  time  were  dif¬ 
ferent  for  the  processing  and  the  monitoring  personnel.  Changes  in  field 
message  length  had  no  effect  on  monitors'  times;  the  times  remained  con¬ 
stant  at  around  130  sec  onds.  The  monitors'  invariant  processing  limes 
were  expected  because  they  receive  processed  data  rathe)-  than  field  mes¬ 
sages. 
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FIELD  MESSAGE  LENGTH 

Figure  14.  Time  per  message  as  a  function  of  field  message  length. 


There  was  a  considerable  effect  of  message  length  on  the  mes¬ 
sage  completion  time  for  the  processing  personnel  (Figure  14).  It  ap¬ 
pears  that  the  limits  of  the  F  MAR  S'  processing  personnel  were  exceeded 
when  message  length  was  between  100  and  1000  characters. 

Further  insight  into  the  point  at  which  message  processing  was 
affected  can  be  obtained  by  examining  the  percentage  of  time  busy  for  the 
various  operators.  This  dependent  measure  is  plotted  against  field  mes-  < 

sage  length  in  Figure  15.  The  trend  of  the  completion  times  was  again 
very  different  across  the  personnel  levels.  The  percentage  lime  busy  for 
processing  per  sonnel  incr  eased  as  the  number  of  characters  increased. 

The  percentage  lime  busy  for  the  monitors  remained  level  at  lower  mes¬ 
sage  lengths  but,  when  message  length  readied  the  highest  value  tested, 
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Figure  15.  Percentage  time  busy  as  a  function  of  message  length. 


the  time  busy  dropped  to  a  few  percent.  The  inability  of  the  simulated 
processing  personnel  to  process  the  extremely  long  messages  (1000  char¬ 
acters)  had  the  effect  of  virtually  stopping  all  flow  into  the  system's 
higher  levels. 

Even  the  increased  stress  on  the  processing  personnel,  as  simu¬ 
lated  in  the  NKTMAN  model,  was  not  sufficient  to  offset  the  effects  of 
1000  character  messages. 

The  effectiveness  measures  also  reflected  the  rather  strong  inhib¬ 
itory  action  of  increased  message  length.  Overall  effectiveness  and  its 
components  are  plotted  against  field  message  length  in  Figure  16.  The 
thoroughness  and  the  responsiveness  components  botli  dropped  from  mod¬ 
erate  or  high  values,  when  the  message  length  was  100  characters,  to  low 
or  moderate  levels  when  message  length  was  1000.  The  result  was  a  par¬ 
allel  depression  of  the  overall  effectiveness  index. 
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Figure  1G.  Overall  effectiveness  and  effectiveness  components  as  a 
function  of  field  message  length. 


A  comparison  of  the  E MARS'  effectiveness  degradation  as  a  func¬ 
tion  of  message  length  with  the  cori'esponding  measure  for  a  TWSEAS- 
type  system  suggested  greater  robustness  for  the  EMARS  system.  The 
pertinent  data  from  both  the  EMARS  and  TWSEAS-type  simulations  arc 
presented  in  Table  4.  Some  differences  in  the  runs  are  noted  here; 

( 1 )  in  the  low  to  middle  range,  EMARS'  field  message  length  was  30  char¬ 
acters  whereas  the  TWSEAS-type 's  was  22,  and  (2)  the  KE'fMAN  evalua¬ 
tion  using  the  TWSEAS-type  structure  did  not  include  any  field  messages 
longer  than  100  characters.  Within  these  limitations,  compar  isons  wer  e 
possible  and  are  pr  esented  in  Table  4. 


Table  4 


The  thoroughness  measure  for  EMAltS  was  observed  to  be  con¬ 
siderably  higher  across  each  comparison.  In  addition,  responsiveness 
was  found  to  be  almost  twice  as  high  in  EMAHS  as  in  the  TWSEAS-type 
system  when  the  length  was  100.  In  fact,  the  responsiveness  measure 
of  EMAHS  was  higher  at  1000  characters  per  field  message  (value  of 
0.70)  than  the  TWSEAS-type  at  100  characters  (value  of  0.47). 

The  change  in  accuracy  over  the  changes  in  message  length  was 
trivial  for  EMAHS.  The  accuracy  values  were  consistently  high  (0.  97 
to  0.  99)  but,  for  the  100  characters  per  message  length,  the  TWSEAS- 
type  system  was  considerably  less  accurate  (0.  74). 

'['he  KMAHS's  showed  a  five  or  six  point  overall  effectiveness  ad¬ 
vantage  over  the  TWSEAS-type  system  when  the  message  length  was  low. 
However,  at  100  characters  per  message,  the  difference  was  considerable. 

Staff  Message  Length 


The  final  physical  variable  examined  the  effects  of  message  length 
to  the  monitors  - -the  EXMOIC  and  EMIC. 

The  monitors' time  per  message  completion  increased  (Figure  17) 
when  message  length  was  greater  than  100  characters.  This  processing 
time  at  700  characters  per  message,  in  turn,  increased  busy  time  from 
about  28  percent  in  the  baseline  to  about  36  percent.  The  effect  of  field 
message  length  on  the  overall  effectiveness  index  or  on  its  components 
was  neglible  except  for  the  thoroughness  component.  Essentially,  the 
same  effect  was  reported  from  the  TWSEAS-type  sensitivity  test  (Siegel 
ct  al.  ,  1979),  i.e.  ,  little  variability  in  effectiveness  measures  as  a  func¬ 
tion  of  staff  message  length.  The  only  effect  of  any  consequence  was  ob¬ 
served  in  the  thoroughness  component.  Thoroughness  values  within  and 
between  systems  were: 
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EMAHS 

f>0 

.  97 

.  98 

100 

.  98 
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Figure  17.  Time  per  message  as  a  function  message  length 
to  monitors. 


ilia  thoroughness  measures  for  the  two  systems  compared  were  equiva¬ 
lent  and  high  across  the  first  two  staff  message  leugths--50  and  100  ehar- 
aclers  in  length.  However,  at  a  length  of  300  c  haracters,  the  TWSEAS- 
lype's  thoroughness  dropped  sharply  to  0.77  whereas  that  for  the  EM  UiS 
decreased  only  slightly  to  0.93.  More  importantly,  staff  message  lengths 
ol  700  characters  only  reduced  the  thoroughness  to  0.  90.  This  value*  is 
higher  than  that  observed  for  the  TWSEAS-tvpe  when  the  length  was  300 
characters.  1 


Behavioral  Variables 


Three  behavioral  variables  were  manipulated  in  the  EM  AHS1  con¬ 
cept  evaluation:  (1)  operator  proficiency,  ( 2 )  operator  precision,  and 
(3)  operator  level  of  aspiration.  The*  effects  of  these  variables  on  system 
pel  forinanre  and  on  the  eflectiveness  index  are  examined  in  this  section. 
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Ope  rator  Speed 

NET  MAN  allows  simulation  of  operators  of  various  proficiency 
(speed)  by  way  of  input  values.  In  the  EMAHS'  c  oncept  simulation,  the 
effects  of  operator  proficiency  were  examined  over  the  range  of  0.7  (con¬ 
siderably  above  average)  to  1.6  (considerably  below  average)  where  1.0 
is  average. 

The  effects  of  varying  simulated  operator  proficiency  on  perform¬ 
ance  are  presented  in  Figure  18  (A)  for  the  number  of  messages  completed 
and  in  Figure  18  (II)  for  the  time  per  message.  The  proficiency  variation 
had  a  linear  effect  on  both  dependent  variables. 

As  would  be  anticipated,  the  number  of  messages  completed  was 
positively  related  to  tile  proficiency  values  assigned  to  the  simulated  op¬ 
erators.  The  reduction  in  messages  completed  with  decreases  in  simu¬ 
lated  operator  proficiency  was  of  moderate  magnitude.  For  the  proces¬ 
sing  personnel,  the  reduction  was  approximately  14  percent,  whereas 
the  reduction  for  the  monitors  was  about  15  percent,  as  proficiency  var¬ 
ied  from  0.  7  to  1.  6. 

As  the  simulated  proficiency  increased,  the  time  per  message  de¬ 
creased.  With  decreasing  proficiency,  there  was  a  tendency  for  the  proc¬ 
essing  personnel's  message  completion  time  to  increase  at  a  faster  rate 
than  did  the  message  completion  time  for  the  monitors.  This  finding  is 
concordant  with  the  prior  implication  of  any  uneven  work  load  across  the 
two  levels.  The  magnitude  of  the  difference  between  levels  was  about  42 
percent  when  the  operators  had  0.7  proficiency  assignments.  This  value 
increased  to  about  100  percent  when  the  simulated  operators  possessed 
1.6  proficiency  assignments. 

The  change  in  time  per  message  was  reflected  in  the  percentage 
time  busy.  Percentage  time  busy  varied  from  23  to  39  percent  and  19  to 
31  percent  when  proficiency  varied  between  0.7  to  1.6  for  the  processing 
personnel  and  the  monitors,  respectively. 

The  overall  effectiveness  index  and  some  of  its  components  also 
suggested  a  rather  direct  effect  of  operator  proficiency  on  system  per¬ 
formance.  Thoroughness,  responsiveness,  and  overall  effectiveness  ex¬ 
hibited  sensitive  changes  in  simulated  operator  proficiency.  When  pro¬ 
ficiency  was  0.  7,  thoroughness  rose  to  0.  94.  This  thoroughness  value 
was  the  highest  observed  throughout,  this  evaluation.  As  proficiency  de¬ 
creased,  thoroughness  decreased  and  was  0.86  when  proficiency  was  1.6. 
In  a  similar  manner,  responsiveness  ranged  from  0.95  to  0.79  as  profi¬ 
ciency  varied  from  0.7  to  1.6.  Overall  effectiveness  exhibited  similar 
changes  and  ranged  between  0.  94  and  0.  09. 
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igure  18.  Messages  completed  (A)  and  the  time  per  message  (B)  as  a  function  of  operator  proficiency. 
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The  effects  attributable  to  manipulations  in  proficiency  were  some¬ 
what  lower  in  magnitude  as  compared  with  the  effects  observed  for  the 
physical  factors.  This  finding  was  to  be  expected  and  is  similar  to  the  re¬ 
sults  reported  in  the  NET  MAN  sensitivity  tests  (Siegel  et  al. ,  1979).  The 
proficiency  variables  had  only  a  limited  impact  on  effectiveness  in  the 
TWSEAS-type  situation.  The  values  and  distributions  of  the  responsive¬ 
ness  and  thoroughness  measures  were  very  similar  between  systems. 


Precision 

Precision,  sealed  similarly  to  proficiency,  represented  the  aspect 
of  human  error  generation.  The  lower  the  precision  value,  the  fewer  the 
number  of  errors  that  would  be  expected  to  enter  the  data  base.  The  ef¬ 
fects  of  precision  on  the  EMARS  were  evaluated  by  examining  four  pre¬ 
cision  values — 0.8,  0.9,  1.0,  and  1.1. 

The  messages  completed  data  and  the  corresponding  time  per  mes¬ 
sage  are  plotted  against  precision  in  Figure  19.  An  effect  very  much  like 
that  reported  for  proficiency  was  found  in  the  relationship  between  mes¬ 
sages  completed  and  precision  [Figure  19(A)].  That  is,  the  number  of 
messages  completed  tended  to  be  linearly  related  to  simulated  precision 
and  similar  distributions  were  evidenced  across  operator  levels.  Across 
operator  levels,  the  degree  of  the  change  was  also  of  the  same  magnitude-- 
about  20  percent. 

The  similarities  between  the  effects  of  proficiency  and  precision 
were  not  maintained  when  the  message  processing  times  were  considered 
[Figure  19  (B)].  The  time  to  process  a  message  was  somewhat  invariant 
when  the  simulated  precision  values  were  between  0.8  and  1.0  for  both 
the  processing  personnel  and  the  monitors.  Thereafter,  time  was  affect¬ 
ed  more  markedly  by  proficiency  than  by  precision.  Time  per  message 
for  processing  personnel  increased  from  approximately  240  and  394  sec¬ 
onds  over  the  precision  values  of  0.8  to  1.  1.  This  represents  a  G4  per¬ 
cent  increase.  The  monitors'  average  processing  time  increase  was 
somewhat  less--4G  percent. 

The  effects  of  precision  on  time  were  reflected  in  percentage  of 
time  busy.  The  highest  percentage  time  busy  was  41  for  the  processing 
personnel  and  29  for  the  monitors. 

The  completeness  and  responsiveness  effectiveness  components  as 
well  as  overall  effectiveness  varied  as  a  function  of  simulated  preci¬ 
sion.  Om;  of  these  measures,  responsiveness,  was  also  found  to  vary 
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as  a  fum-lion  of  prec  ision  in  the  TWSEAS-type  sensitivity  test  (Siegel  el 
al.  ,  11)79).  A  comparison  of  the  effects  of  simulated  operator  precision 
on  the  responsiveness  component  for  the  EMAKS  and  the  TWSEAS-type 
systems  is  presented  below; 


Responsiveness 


Precision 

EMAKS 

TWSEAS-type 

0.8  (high) 

0.  92 

— 

0.9 

0.  92 

6.  92 

1.0 

0.  92 

O 

CO 

© 

1.  1  (low) 

0.  78 

0.  50 

While  the  precision  value  for  more  precise  operators  was  gener¬ 
ally  high  for  both  systems,  the  magnitude  of  the  drop  was  greater  for  the 
TWSEAS-type  system  than  for  the  EMAKS  when  less  precise  simulated 
operators  were  involved.  The  responsiveness  for  the  former  was  0.50 
and  that  for  the  latter  0.78  when  precision  was  1.  1  (low). 

The  completeness  component  also  varied  as  simulated  operator 
precision  was  varied.  Completeness  of  the  EMAKS  system,  measured  as 
the  ratio  of  successfully  completed  task  elements  to  the  total  number  of 
elements,  acted  very  much  like  responsiveness.  The  completeness  index 
was  high  and  stable  over  the  moderate  and  high  simulated  operator  pre¬ 
cision  levels  but  decreased  when  simulated  prec  ision  was  1.  1  (low).  The 
drop  was  from  an  average  of  0.  95  to  a  level  of  0.  82. 


Aspiration 

The  effect  of  operator's  level  of  aspiration  depends,  in  the  NET  MAN 
model,  on  a  complex  interaction  with  the  current  levels  of  performance  and 
stress  on  the  simulated  operators.  Simulated  performance  is  affected  undei 
four  conditions  in  NETMAN; 

1.  when  the  simulated  personnel  are  perform¬ 
ing  below  their  level  of  aspiration  and  are 
only  mildly  stressed,  there  is  a  positive 
pace  adjustment  facto)-  due  to  aspiration. 

2.  when  Ihe  simulated  personnel  are  perform¬ 
ing  above  their  level  of  aspiration  anti  stress 
is  low,  there  is  a  decrease  in  the  operator's 
level  of  aspiration. 
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3.  when  the  simulated  personnel  are  perform¬ 
ing  below  their  level  of  aspiration  but  are 
under  high  stress,  the  level  of  aspiration 
is  reduced  and  the  speed  of  performance  is 
degraded. 

4.  when  the  personnel  are  performing  at  their 
level  of  aspiration  but  are  under  high  stress 
there  is  an  increase  in  stress. 

Aspiration  level,  with  a  range  of  0.8  to  1.0,  is  scaled  directly  in 
NETMAN.  That  is,  higher  values  reflect  higher  aspiration  levels.  To 
evaluate  the  effects  of  operator  aspiration  on  EMAHS'  performance,  in 
separate  runs,  operator  aspiration  was  set  at:  0.8,  0.9,  0.95,  and  0.99. 

Varying  the  simulated  aspiration  level  of  the  simulated  operators 
had  a  moderate  effect  on  both  system  performance  and  effectiveness. 
There  was  an  increase  in  the  number  of  messages  completed  (about  10 
percent)  and  in  completeness  (about  5  percent). 

Examination  of  the  final  aspiration  levels  indicated  them  to  be  such 
as  to  be  active  within  the  model.  The  assigned  and  final  aspiration  levels 
were; 


Aspiration  Levels 


Final 


Assigned 

Processing  Personnel 

Monitors 

0.80 

.  85 

CO 

00 

0.  90 

.  92 

.91 

0.95 

.  90 

.95 

0.  99 

.  99 

.  99 

42 


CHAPTER  IV 


DISCUSSION  AND  CONCLUSIONS 


The  evaluation  of  the  EMARS  concept  employing  the  NETMAN 
model  progressed  along  several  dimensions.  One  dimension  was  con¬ 
cerned  with  the  practical  aspects  of  the  EMARS  system  and  its  limits 
and  strengths.  Another  dimension  was  concerned  with  a  comparison  of 
EMARS'  performance  and  effectiveness  with  that  of  TWSEAS-type  systems. 
The  final  dimension  was  concerned  with  evaluating  NETMAN 'S  effective¬ 
ness  and  adaptability  as  a  field  exercise  monitoring  and  control  system 
evaluation  technique. 


EMARS 


Evaluating  the  EMARS  system  through  employment  of  the  NETMAN 
model  required  the  development  of  a  unique  data  base  and  task  analysis 
along  with  the  use  of  some  of  the  NETMAN  model's  special  features.  The 
evaluation  progressed  over  a  parametrically  manipulated  set  of  variables 
grouped  into  two  classes--physical. and  psychological.  The  results  pro¬ 
vided  unique  insight  into  the  processing  capability  of  EMARS  and  its  strengths 
and  weaknesses.  The  simulated  EMARS  system  was  capable  of  handling  a 
very  high  message  frequency.  The  effects  of  varying  field  message  length 
were  indicative  of  the  limits  of  the  system.  Field  message  lengths  over 
100  characters  or  message  frequencies  over  10  per  operator  per  hour  were 
disruptive  to  EMARS. 

High  staff  message  lengths  did  not  induce  the  same  disruptive  ef¬ 
fects  as  field  message  lengths.  The  results  suggested  that  the  EMARS1 
effectiveness  could  also  be  improved  with  the  addition  of  one  monitor. 

Manipulation  of  psychological  factors  indicated  moderate  operator 
effects.  Operator  proficiency  and  precision  had  linear  effects  on  system 
performance.  System  thoroughness,  responsiveness,  and  completeness 
were  found  sensitive  to  simulated  proficiency  and/or  precision.  Accord¬ 
ingly,  some  training  requirements  seem  required  for  EMARS1  personnel. 

TWSEAS 


When  it  was  possible,  the  performance  and  the  effectiveness  meas¬ 
ures  obtained  from  the  EMARS1  evaluation  wore  compared  with  prior  sim¬ 
ulations  of  a  TWSEAS-type  system. 
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Comparing  the  EMAHS  with  the  TWSEAS- type  system  indiealed  the 
EMARS  to  be  more  robust  and  to  possess  a  greater  capacity. 

Across  systems,  comparisons  rather  consistently  suggested  that 
the  EMAHS  concept  was  superior  on  a  variety  of  measures.  The  most 
pronounced  differences  were  observed  across  message  frequency.  The 
overall  effectiveness  and  the  thoroughness  of  the  TWSEAS-type  system 
were  much  more  severely  degraded  than  the  EMARS  as  message  frequen¬ 
cy  increased.  In  fact,  the  decline  for  EMARS  was  somewhat  shallow. 

Rather  large  differences  between  the  two  system  types  were  also 

indicated  when  field  message  lengths  were  varied _ Even  where  EMARS' 

data  suggested  considerable  inability  to  process  very  long  field  message 
lengths,  the  EMARS  simulation  indicated  less  degradation  and  EMARS  per¬ 
formed  comparatively  better. 

In  terms  of  the  psychological  variables,  for  which  differences  be¬ 
tween  the  systems  were  not  as  pronounced,  the  EMARS  system  handled 
messages  more  effectively  across  changes  in  simulated  operator  precision. 

Overall,  it  appears  that  the  EMARS  concept  possesses  advantages 
over  the  TWSEAS-type  system  in  situations  in  which  work  load  is  expected 
to  be  moderate  to  high. 


NETMAN  Evaluation 

One  of  the  purposes  of  the  present  work  was  to  test  further  the 
capability  or  utility  of  the  NETMAN  model.  The  test  was  demonstrative 
rather  than  quantitative  in  nature.  The  use  of  NETMAN  to  evaluate  the 
EMARS  demonstrates  NETMAN 's  adaptability  by  applying  NETMAN  to  a 
novel  communication  network  and  its  personnel.  The  results  yielded  a 
variety  of  insights  relative  to  EMARS'  design  and  the  results  were  rational 
in  terms  of  direction  of  change  with  variation  in  input  parameters.  More¬ 
over,  NETMAN  was  sensitive  to  differences  in  design  characteristics  and 
performance  capabilities  among  the  EMARS  and  TWSEAS-type  systems. 

NETMAN 's  sensitivity  to  differences  in  the  simulated  systems 
suggests  that  computer  modeling  of  field  exercise  monitoring  and  con¬ 
trol  systems  may  be  a  reasonable  way  to  test  economically  and  effic¬ 
iently  the  capabilities  of  such  systems  while  they  arc  in  the  conceptual 
stage  of  development.  Moreover,  it  seems  that  such  evaluations  may 
be  completed  with  some  degree  of  precision  and  some  assurance  that  the 


BEST  AVAILABLE  COPY 


44 


results  will  possess  meaningful  implications  for  equipment  and  personnel 
subsystem  design. 

The  results  of  the  earlier  sensitivity  and  validation  tests  (Siegel 
et  al.  ,  1979)  of  NETMAN,  along  with  the  current,  findings,  combine  to  sug¬ 
gest  that  NETMAN  is  a  useful  evaluation  tool.  Its  validity  was  previously 
demonstrated  and  contentions  favoring  its  flexibility  and  adaptability  are 
supported  by  the  present  work. 
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GLOSSARY 


accuracy 


ccc 


completeness 


controller 


effectiveness 


EMARS 


execution  time 


exercise  monitoring 
and  reporting  system 

I/O  device  operator 


iteration 


a  component  of  effectiveness,  one  minus  the  quo¬ 
tient:  total  information  loss/ number  of  messages 
completed. 

designation  for  the  central  computer  which  receives 
messages,  processes  them,  and  decodes  them  for 
presentation  to  the  controllers. 

the  average  of  all  operator  performance  values  taken 
over  all  men  simulated  (i.  e. ,  the  percentage  of  task 
elements  performed  successfully).  A  component  of 
the  effectiveness. 

personnel  who  receive  field  information  generated 
by  referees,  sent  by  radio  operators,  and  decoded 
by  the  computers  and  who  use  this  information  to 
organize  and  conduct  military  field  exercises. 

measure  of  the  capability  of  a  military  unit  to  per¬ 
form.  A  variable  determined  as  a  function  of  the 
thoroughness,  completeness,  responsiveness,  and 
accuracy  variables. 

the  concept  for  a  system  whose  purpose  is  to  monitor 
a  military  exercise,  to  assess  performance  of  the 
military  personnel,  and  to  report  the  results  of  that 
assessment. 

the  value  calculated  by  a  simulation  model  to  represent 
the  actual  duration  of  performance  by  the  simulated 
operator(s)  under  current  conditions. 

a  collection  of  equipment,  personnel  procedures,  and 
support  capable  of  assessing  proficiency  of  a  military 
field  exercise  semiautomatically. 

a  person  responsible  for  operation  of  an  input/output 
device,  i.  e. ,  computer  peripheral  device  such  as  a 
terminal. 

the  number  of  times  a  simulation  situation  is  re¬ 
peated  in  a  model  to  level  out  stochastic  effects. 
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message 


message  completion 


NETMAN 


operator  aspiration 


operator  fatigue 


operator  performance 


operator  precision 


operator  speed 


operator  stress 


radio  operator 


responsiveness 


a  series  of  alphanumeric  characters,  formulated 
in  accordance  with  a  defined  procedure  which  can 
be  transmitted  between  persons/computers  simu¬ 
lated  by  a  model. 

the  completion  of  processing  of  a  message  by  all 
operators  (and  CCC) which  processes  it  in  sequence. 

a  computer  simulation  model  for  evaluation  of  mil¬ 
itary  field  exercise  management  systems  in  which 
the  processing  of  each  message  by  each  person  is 
simulated. 

the  value  of  operator  precision  to  which  a  simu¬ 
lated  operator  aspires;  represents  a  person's  goals, 
expectation,  or  motivation  level. 

a  measure  of  a  simulated  operator's  current  level 
of  physical  exhaustion  as  a  result  of  the  amount  of 
work,  rest,  and/or  sleep  he  has  experienced. 

the  percentage  of  tasks  completed  successfully  for 
a  given  operator. 

a  measure  of  a  simulated  operator's  rate  of  error 
in  performing  tasks.  (High  precision  is  lower  value 
in  NETMAN.  ) 

a  measure  of  the  relative  time  in  which  a  person 
can  perform  tasks  which  are  predominantly  motor 
oriented,  closely  related  to  operator  proficiency. 

a  measure  of  the  pressure  perceived  by  one  or 
more  operator(s)to  complete  the  remaining  sim¬ 
ulated  subtasks;  a  measure  of  simulated  operator 
state  of  mind  involving  a  threat  or  negative  impact. 

an  operator  who  processes  messages  for  entry  via 
his  radio  on  a  "first -in  first-out" basis  as  they  are 
completed  by  the  referees. 

a  functional  component  of  effectiveness;  a  ratio  of 
average  message  processing  time  (i.  e. ,  referee 
start  to  controller  end)  to  average  total  elapsed 
message  time  including  idle  periods. 
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stimulus  message 


a  message  entered  into  the  system.  A  stimulus 
message  may  generate  one  or  more  messages  to 
be  processed  within  the  system. 


simulated  message 


simulated  model 


stochastic  model 


subtask  procedure 


system  entry  message 
or  stimulus  message 


task  analysis 


thoroughness 


TWSEAS-like  system 


a  vector  whose  elements  represent  the  primary 
characteristics  of  a  military  message  (e.  g.  ,  length 
in  characters,  priority,  type,  and  ID  number. 

a  representation  of  behavior  and  behavioral  influ¬ 
ence  implemented  on  a  digital  computer  so  as  to 
allow  prediction  of  an  event  or  event  sets. 

a  simulation  model  containing  Monte  Carlo  or  game- 
of -chance  techniques  involving  the  use  of  pseudo¬ 
random  numbers  in  the  calculations  of  current  mod¬ 
el  variables. 

a  number  identifying  a  subtask  which  must  be  suc¬ 
cessfully  completed  prior  to  initiating  work  on 
(simulation  of)  a  given  subtask. 

a  message  which  enters  the  simulated  system  from 
outside  the  system  and  which,  upon  entry,  generates 
one  or  more  messages  in  the  simulated  system. 

a  sequential  list  of  subtasks  which  comprise  a  task; 
includes  data  and  information  describing  the  per¬ 
formance  characteristics  of  each  subtask. 

the  ratio  of  the  number  of  messages  completed  per 
hour  to  the  total  number  arriving  during  the  hour. 

A  component  of  effectiveness. 

a  system  in  which  the  message  generation  and  proc¬ 
essing  by  operators  and  computer  is  handled  in  a 
way  similar  to  that  utilized  in  the  TWSEAS  military 
system. 


